BMC Medical Education
○ Springer Science and Business Media LLC
Preprints posted in the last 90 days, ranked by how well they match BMC Medical Education's content profile, based on 21 papers previously published here. The average preprint has a 0.06% match score for this journal, so anything above that is already an above-average fit.
Sekine, M.; Nishizaki, Y.; Watari, T.; Shikino, K.; Fukui, S.; Nagasaki, K.; Nojima, M.; Shimizu, T.; Yamamoto, Y.; Kobayashi, H.; Tokuda, Y.
Show abstract
Introduction: Postgraduate clinical training is crucial for developing professional competence, communication skills, and effective teamwork. Although resident physician selection is crucial, little is known about how Japanese residency programs select residents and whether selection practices are associated with difficulties during training. Methods: We conducted a nationwide cross-sectional survey of residency programs participating in Japan's 2023 General Medicine In-Training Examination (GM-ITE). Program directors completed a questionnaire assessing selection methods, interview content, quality-assurance measures, and resident difficulties, defined as at least one postgraduate year 1 or 2 resident physician receiving disciplinary action or a severe warning. Free-text responses were coded using the Situation, Task, Action, and Result framework. Associations between selection methods and resident difficulties were examined using adjusted logistic regression models controlling for hospital type and number of GM-ITE examinees. Results: Of 151 participating physician-selection programs, 150 provided valid responses. Interviews were used by 90.1% of programs and were identified as the most important selection component by 87.3%. Thirty-five programs (23.3%) reported difficulties with resident physicians, involving professionalism and workplace conduct including rule, ethics, or boundary violations, work avoidance or unavailability, and inappropriate communication. Use of applicants' pre-clinical-clerkship computer-based test scores as a selection criterion was associated with resident difficulties (adjusted OR, 4.60; 95% CI, 1.50-14.11; P = 0.008; FDR-adjusted P = 0.048). No significant associations were observed for essays, academic tests, medical school grades, or personality assessment. Program-level GM-ITE total and domain scores did not differ significantly between programs with and without reported resident physician difficulties. Discussion: Resident physician selection in Japan is highly interview-centered; reported difficulties were more often related to professionalism and workplace conduct than to knowledge deficits. Although these exploratory findings are program-level, they highlight the importance of strengthening the quality assurance processes in resident selection systems, particularly for assessing professionalism-related attributes in applicants.
Kadam, C. Y.; Dhok, A.; Khanna, K.; Waghmare, T.
Show abstract
Background and objective: Problem-based learning (PBL) strategies such as case studies and role-playing are increasingly adopted in medical education to promote critical thinking, communication, and teamwork. However, comparative evidence regarding their effectiveness in medical biochemistry remains limited. This study aims to evaluate and compare the impact of case studies and role-playing on learning outcomes, engagement and skill development among first year MBBS students. Material and methods: An educational randomized controlled trial (RCT) with a within-subjects crossover design was conducted in the Department of Biochemistry involving forty-five first-year MBBS students. Participants were randomly assigned to receive case studies and role-playing in a counterbalanced sequence. Learning outcomes were assessed using pre- and post- tests, a structured feedback questionnaire and qualitative reflections. Data were analysed using appropriate parametric and non-parametric tests, along with adjusted analysis using ANCOVA to account for baseline performance. Results: All participants completed both methods, with no significant difference in baseline knowledge between arms (P=0.12). Both case studies and role-playing produced significant pre- to post- test improvements (P<0.001). Unadjusted analyses showed higher learning gains with role-playing, but pre-crossover ANCOVA demonstrated comparable effectiveness after baseline adjustment. Exploratory pooled analysis suggested a moderate cumulative advantage for role-playing. Student feedback indicated higher engagement and perceived skill development with role-playing, while both methods were viewed as relevant and educationally valuable. Conclusion: Both case studies and role-playing were effective PBL strategies for improving understanding and application of medical biochemistry concepts. Although student feedback and unadjusted analyses indicated higher engagement with role-playing, adjusted analysis at initial exposure showed comparable effectiveness between the two methods. Role-playing supported interactive learning and AETCOM -related skills, while case studies strengthened structured analytical reasoning. A combined approach may therefore offer a balanced and comprehensive PBL framework in medical education.
McGurn, C.; George, L.
Show abstract
Objectives: This study aimed to examine physiotherapy students reaction and learning, following Simulation Based Education (SBE) during a Cardiorespiratory module. A further aim was to see if any learning translated into clinical placement. Design: A mixed methods research design consisting of a questionnaire (Phase1) after the activity which was underpinned by the Kirkpatrick model of evaluation. This was followed by a focus groups (Phase 2) after completion of clinical placement. Participants: 92 final year physiotherapy students at a single institution were eligible to participate in the SBE session with n=81 (88%) students completing the survey, and 8 students participating across 2 focus group sessions. Results: Survey: Over 80% of students strongly agreed on a positive initial reaction to SBE. Learning yielded a 76% and above response of strongly agree in all areas except confidence. Students valued SBE as a preferred learning and teaching strategy and wanted more. They welcomed SBE as a supplement but not substitute for clinical placement. Students felt skills learned could be transferred into all areas of clinical practice,, namely communication and decision making. Conclusion: Students rated SBE positively with development of transferable non-technical skills. Reaction to SBE was high in terms of relevance, engagement and satisfaction. Self-perceived confidence, although positive, was the comparatively lowest scoring of the domains. Students advocated SBE as a supplement rather than a substitute for clinical placement preferring a hybrid approach. Contribution of the Paper: Adds to the positive body of evidence which already exists towards SBE, especially in the development of non-technical skills. Suggestions are made that SBE cant make students feel fully confident in preparation for clinical placement. While students in this study expressed that they wanted more SBE and earlier, this paper found that students would not advocate SBE replacing clinical placement.
Impito, P. F.
Show abstract
Medical education in resource-limited settings faces significant challenges in providing diverse clinical exposure and fostering essential skills such as clinical reasoning, communication, and empathy. Due to the inability to afford immersive technologies such as virtual reality (VR) and Augmented Reality (AR), constrained by financial, infrastructural, and structural barriers, interactive simulation videos (ISV) constitute an innovative, cost-effective educational tool that can bridge the gap between theoretical knowledge and practical clinical experience, while enhancing student engagement and learning outcomes. This study aimed to assess the educational value and user experience of ISV as a supplementary tool in teaching medical semiology among medical students in Mozambique. A quantitative, descriptive, cross-sectional study was conducted among 4th-year medical students at Alberto Chipande University. Descriptive and inferential statistical analyses were performed, including a one-sample t-test to compare responses against a value. A total of 93 students participated in the study. The ISV was highly rated for realism (77.4%), relevance to training (74%), and usefulness of feedback (78.5%). Most students reported increased confidence in patient care (81.7) and found the digital patient credible and engaging (74.2). Overall mean scores across all domains were significantly higher than the neutral benchmark (p<0.05), indicating a positive perception of the tool. Students also expressed a strong willingness to recommend its integration into medical curricula. In conclusion, ISV represents a valuable and feasible pedagogical approach in medical education, particularly in low-resource settings. They enhance clinical reasoning, engagement, and confidence while providing scalable, standardized learning experiences. ISV holds strong potential as a complementary tool to bridge gaps in traditional medical training and improve educational equity.
Lopez, A.; Gabellier, L.; Peytavi, C.; Loi, Z.; Poupin-Lavigne, E.; Plotton, C.; Dupont, V.; Jahdauti, L.; de Vries, P.; Thuillier, P.; Compagnat, M.; Vergne-Salle, P.; Kuchenbuch, M.; Feigerlova, E.; Berthelot, P.; Bednarek, N.; Cochener-Lamard, B.; Robert, P.-Y.; Braun, M.; Morin, D.; Deruelle, P.; Laffont, I.; Pers, Y.-M.; Yauy, K.
Show abstract
BACKGROUND: As artificial intelligence (AI) enters clinical practice, concerns about deskilling and never-skilling make the independent assessment of clinical competence increasingly important. The AI-Standardized Clinical Examination (ASCE) uses AI to simulate and score standardized clinical encounters, potentially overcoming some of the logistical and standardization constraints of conventional objective structured clinical examinations (OSCEs). Whether it provides valid and reliable assessment at scale is unknown. METHODS: In this prospective multicenter cohort study, final-year medical students from six French medical schools completed a 10-station ASCE six weeks before the national OSCE session. GPT-4o simulated standardized clinical partners; transcripts were subsequently scored by GPT-4.1 against faculty-defined rubrics. The primary outcome was the correlation between ASCE and national OSCE scores. Secondary outcomes were inter-rater reliability of expert transcript scoring, alignment of ASCE scores with expert ratings, and internal consistency. RESULTS: Among 948 enrolled students, 588 had paired ASCE and national OSCE scores. ASCE scores were correlated with national OSCE performance (Spearman's {rho}, 0.57; 95% CI, 0.51 to 0.63), with less than 1% of residual variance attributable to center (intraclass correlation coefficient, 0.01). Eight experts scored 20 randomly selected ASCE transcripts; Krippendorff's was 0.77 overall, 0.80 for clinical skills, and 0.70 for communication and attitude skills. Adding GPT-4.1 as an evaluator reduced by 0.02, and its scores did not differ significantly from those of individual experts. Internal consistency was 0.67 across 10 stations and was projected to approach 0.80 with 20 stations. CONCLUSIONS: Scores from an examination delivered and scored by AI were associated with performance on a national high-stakes OSCE, with minimal center-level variation, tentative agreement among expert raters, and automated scores that aligned with expert ratings. These results support AI delivery and scoring of clinical examinations, with expert review of automated scores.
Rothermich, K.; Clark-Stallkamp, R.; Baker-Iyore, R. P.; Cuatico, C.; Lee, J. G. L.; Bobb, S. C.
Show abstract
Objectives: Health professions students often receive limited training on how to adjust their communication for patients whose first language is not English (LX), despite the risks for diagnosis, treatment, and patient trust. Based on Communication Accommodation Theory, we developed a brief module to improve students' awareness and knowledge of strategic communication adjustments for LX patients. Methods: We tested feasibility in a single-group, pre-post pilot study guided by the Kirkpatrick framework with undergraduate (n=22) and graduate (n=41) speech-language pathology and psychology students in Massachusetts and North Carolina. The module included instructional content, video and audio examples, interactive activities, and a podcast. Learning was assessed with knowledge checks, and self-reported facilitative and resistant accommodation with the Communication Accommodation Behavior Scale (CABS). After training, students rated their confidence and likelihood of using accommodation strategies. Results: At both sites, facilitative accommodation increased and resistant accommodation decreased significantly, with a larger decrease at the undergraduate site. Knowledge-check accuracy was high at both time points, suggesting a ceiling effect. Students reported moderate- to-high confidence in adapting their speech and high likelihood of using a learned strategy in future clinical encounters. Conclusions: A brief module can be feasibly delivered across institutions and is associated with short-term improvements in self-reported communication accommodation. Practice implications: We recommend future work evaluate whether these changes translate into clinical behavior long-term and refine knowledge-check items to better capture learning gains.
Hasan, S.; Tang, G. H.; Van Bakel, T.; Fralick, M.; Sholzberg, M.
Show abstract
Background: Parenthood in medicine is an experience fraught with challenges that begin in the pre-conception phase and persist as physician-parents navigate their dual role. Methods: This study aims to understand the experiences of physician-parents (both childbearing and non-childbearing) returning to work after parental leave. Physicians, including residents, affiliated with the University of Toronto Department of Medicine internal medicine division who had a child within the last ten years were invited to participate in a self-administered electronic survey from June2022 to 30June 2023. Data were summarized using descriptive statistics. Results: Twenty-nine respondents took part in the survey; 79% were staff physicians while the remaining were trainees at the time of the survey. At the time their first child was born, 48% were staff members and 52% were trainees. The median duration of parental leave was 5 months (range 1-12 months). Among those for whom lactation was applicable, 76% reported that they continued their lactation efforts on returning to work, while 24% chose not to. Thirty five percent (35%, n=6/17) of respondents cited that they were not provided with duty hour modifications when required. Many respondents (46%) felt guilty about burdening their co-workers for requesting accommodations for childcare. When requested to rate their experiences of balancing their personal and professional lives on a scale of 1-10 (1 being impossible to maintain and 10 being extremely easy to maintain), most respondents (66%) chose a rating of 5 or less. Conclusion: This study supports previous findings that physician-parents face a myriad of challenges integrating their roles in medicine and parenthood. A change in culture is required to support and normalize parenthood for physicians and foster sustainability and productivity in academic medicine.
Zhang, L.; Hou, Y.; Li, B.; Wu, K.; Zhang, j.; Yang, M.
Show abstract
ObjectiveTo establish a standardized training program for endoscopic pathogen visualization literacy (EPVL) based on fluorescence rapid on-site evaluation (ROSE) technology for gastroenterologists, and to evaluate its training efficacy. MethodsA prospective quasi-experimental study was conducted. A total of 54 gastroenterology trainees were non-randomly allocated into the EPVL training group (Group A, n=28, 16-hour comprehensive training) and the control group (Group B, n=26, 3.5-hour traditional teaching). Pre- and post-training assessments included theoretical examinations, fluorescence ROSE image interpretation tests (30 parallel images per set), interpretation speed measurement, and clinical decision-making integration evaluation. The primary outcome was the change in image interpretation accuracy, analyzed by ANCOVA with pre-test scores as the covariate. ResultsBaseline characteristics were comparable between groups (P>0.05 for all demographic variables and pre-test scores). Group A showed significant improvement in image interpretation accuracy from 57.8{+/-}13.6% pre-training to 82.5{+/-}11.2% post-training (improvement of 24.7%, paired t=-12.86, P<0.001), while Group B improved from 58.5{+/-}13.0% to 71.0{+/-}13.5% (improvement of 12.5%, paired t=-5.24, P<0.001). After ANCOVA adjustment for pre-test scores, the between-group difference was significant (F(1, 51)=10.95, P=0.0017, 2=0.177), with Cohens d=0.94 (large effect size). Interpretation speed in Group A (19.2{+/-}2.8 s/image) was significantly faster than in Group B (32.5{+/-}6.0 s/image, t=-10.45, P<0.001). Clinical decision-making scores were significantly higher in Group A (80.5{+/-}8.0 vs. 65.3{+/-}11.5, t=5.60, P<0.001). The Kappa agreement with the gold standard in Group A improved from 0.56{+/-}0.18 to 0.84{+/-}0.11 (t=-8.35, P<0.001). Participant satisfaction exceeded 88%. ConclusionThe EPVL training program significantly improves gastroenterologists fluorescence ROSE image interpretation accuracy, speed, and clinical decision-making integration, providing a novel and effective standardized training paradigm for digestive endoscopy education.
Preiksaitis, C. M.; Makutonin, M.; Abu-Jubara, D.; Chang, W.-T. W.; Cooney, R.; Diercks, D.; Kundurti, N.; Kwan, B.; Liu, R.; Lozano, M.; Mackey, C.; Moukaddem, M.; Pearce, E.; Pelletier, J.; Rowh, M.; Taylor, R. A.; Rose, C.
Show abstract
Artificial intelligence (AI) is increasingly embedded in U.S. emergency department workflows, but no medical specialty has defined what competent use of these tools requires of its physicians. Existing accreditation requirements are silent on AI, and parallel national efforts span the learning continuum but are intentionally specialty-agnostic. As one output of a working group of the 2026 Society for Academic Emergency Medicine (SAEM) Artificial Intelligence Consensus Conference, we developed an emergency medicine-specific framework for AI competency. We employed a multiphase consensus design: a Nominal Group Technique session generated themes from a curated set of clinical AI scenarios, and a two-round modified Delphi process (panel n = 13 across 12 academic medical centers), separated by a consolidation videoconference, evaluated and extended the framework. A priori consensus required [≥]70% of panelists rating [≥]4 on a 5-point Likert scale ([≥]75% for themes) with interquartile range [≤]1; we followed established Delphi reporting guidance. The framework comprises 5 themes (Communicating about AI, Understanding appropriate use cases, Interacting with AI, AI risk management, Cognitive impacts of AI), 5 derived competencies (one-to-one theme-to-competency mapping endorsed by 12 of 13 panelists), 19 subthemes, and 10 retained clinical scenarios. All themes, derived competencies, and rated subthemes met consensus thresholds; 9 of 10 scenarios reached consensus, with 1 retained as a future-state operational model. Two cross-cutting conceptual frames emerged: pre-emptive versus post-hoc AI integration, and tiered competencies as an articulated need rather than a pre-specified answer. This is, to our knowledge, the first specialty-specific AI competency framework for a United States medical specialty with quantitative content validity evidence from expert consensus. It provides a structural target for emergency medicine curriculum development, assessment design, and faculty development, with downstream priorities including stage-specific competency assignment, assessment instrument development, and periodic re-evaluation as AI deployment evolves.
Ladur, A. N.; Egere, U.; Murray, C.; Eyinda, M.; Muchemi, O. M.; Suleiman, Z.; Mdoe, M.; Rweyemamu, M. A.; Bello, A. B.; Mohammed, H.; Ameh, C. A.
Show abstract
Background Good quality maternity care is critical in reducing maternal morbidity and mortality in regions with a high maternal and perinatal mortality. Building capacity of maternity care providers through in service training has been proven as effective in bridging the knowledge, competency, and skills gap during provision of maternity care. Meaningful involvement of stakeholder perspectives in the design and implementation of public health training programs heightens the prospects of achieving long term changes in practice and policy. Methods To explore stakeholder perspectives and experiences on the co-development of the antenatal-postnatal care course in Kenya, Nigeria, Tanzania. Data was collected through nine key informant interviews and observation notes between July - October 2022. Qualitative data was analysed in NVIVO software using inductive thematic analysis. Results Study findings showed that stakeholders were receptive of the blended learning approach for training in antenatal and postnatal care describing it as accessible, and effective in improving knowledge and skills. Course facilitators and healthcare providers valued the interactive format, opportunities for pre-session preparation, and continuous support, which contributed to improved learning experiences and collaboration. Conclusion This study highlighted the importance of involving stakeholders in the design and implementation of the antenatal-postnatal blended learning course in three countries. Eliciting feedback based on stakeholder experiences and incorporating it in real time strengthened the implementation of the blended learning course.
Bhagwagar, S. Z.; Abushouk, A.; Dempsey, J.; Brock, D. C.; Obradovic, A.; Faraz, A.; Cross, J. L.; Siebert, A.; Tun, H. N.; Lin, K. M.; Wilson, E. M.; Tang, C. Y.; Ghosh, A.; Nguyen, M.; Jha, A.; Shalev, D.; Noch, E.; Kwan, J. M.
Show abstract
Importance: Physician-scientist careers depend on sustained research funding, protected time, and institutional resources. However, data on career-retention attitudes across career stages are limited. Objective: To characterize career intentions and associated research-environment factors among MD-PhD students and early-career physician-scientists. Design: Two independent, national cross-sectional surveys were conducted during nonoverlapping periods in 2025; MD-PhD applicant and matriculant trends were also examined. Setting: Medical schools and academic health centers in the United States. Participants: MD-PhD students and early-career physician-scientists (ECPS). Exposures: MD-PhD exposures included perceived institutional support, training-related mental health impact, and research disruption. ECPS exposures included institutional support, adverse funding-related wellbeing, and direct National Institutes of Health funding-process disruption. Main Outcome and Measures: Primary outcomes were consideration of leaving the physician-scientist pathway among MD-PhD students and consideration of leaving academic medicine or research among ECPS. Secondary outcomes included likelihood of staying in academic medicine, research-intensive career preference, and consideration for leaving the US to continue research. Results: Among MD-PhD programs confirmed to have distributed the survey, 838 student responses were received (estimated response rate of 19.3%); 175 ECPS responses were received, with no calculable individual-level response rate. Among MD-PhD students, 423 of 835 (50.7%) reported considering leaving the physician-scientist pathway; 712 of 837 (85.1%) reported a high likelihood of remaining in academic medicine, and 664 of 836 (79.4%) preferred a research-intensive career. Adequate institutional support was associated with lower odds of considering leaving (adjusted odds ratio [aOR], 0.28; 95% CI, 0.14-0.54) and higher odds of staying in academic medicine (aOR, 5.65; 95% CI, 2.91-10.98). Training-related mental health impact (aOR, 2.34; 95% CI, 1.72-3.19) and research disruption (aOR, 1.83; 95% CI, 1.35-2.48) were associated with greater odds of considering leaving. Among ECPS, 57.7% considered leaving, 52.6% reported a high likelihood of staying, and 42.9% considered leaving the United States to continue research. Adverse funding-related well-being was associated with considering leaving academic medicine (aOR, 3.13; 95% CI, 1.52-6.63) and leaving the United States to continue research (aOR, 5.29; 95% CI, 2.35-13.12). Conclusions and Relevance: Consideration of leaving was common despite continued interest in academic and research-intensive careers. Institutional support, adverse well-being, and research disruptions were associated with career intentions.
Joseph-Delaffon, K.; Desgrouas, M.; Catanese, S.; Lejeune, J.; Nait-Kaci, J.; Piver, E.; Breteau, I.; Leducq, S.; Gatault, P.; Khanna, R. K.; Angoulvant, D.; Vallet, N.
Show abstract
Background. Designing high-quality Objective Structured Clinical Examination (OSCE) stations is a time-consuming process. Generative artificial intelligence (AI) represents a promising path to accelerate content creation by automating the generation of scenarios. A growing number of AI tools is now available for this purpose. Objective. To assess the variability between generative AI models in their ability to produce OSCE stations in the field of paediatrics. Methods. A structured prompt was developed based on the French national OSCE guidelines for medical education. Five distinct AI models were provided with this prompt, alongside the neonatal jaundice chapter from the French pediatric reference textbook, to generate 6 complete OSCE stations. Results. Prompt compliance was high for ChatGPT 5.1, ChatGPT 5.2, Gemini 3.0 Pro, and Claude Opus 4.5, while it was lower for Grok 4.1. Expert-rated quality was generally high, with few factual errors or missing information across models. However usability differed significantly between models. This was also true for several quality dimensions such as checklist clarity, embedding of checklist answers within vignettes, and ease of standardized patient formation. ChatGPT 5.1 required the most revisions and Gemini most often rated usable as is. Significant inter-model differences were observed in diagnostics, only with ChatGPT 5.1 sampling all three neonatal jaundice categories. Contextual variables showed systematic narrowing across models. Clinical grid density was consistent (10-12 items per station), but thematic distribution differed markedly. Soft skills coverage varied significantly across models (p=0.002), none of them consistently representing all communication competency domains. Conclusion. Large language models can generate structurally compliant OSCE stations, but surface compliance conceals substantive inter-model differences in diagnostic coverage, contextual diversity, and soft skills representation, that compromise content validity. No model currently meets the criteria for unsupervised deployment in a summative assessment bank. The choice of model carries pedagogical implications and expert curation remains essential before integration into high-stakes assessment workflows.
Otte, J. H.; Cartagena, A.
Show abstract
Background. A primary constraint on the capacity of EMS programs to meet industry demand is psychomotor instruction and verification, requiring direct observation of each student by a qualified evaluator. Whether AI video analysis can relieve it is untested; none has been applied to EMS skill examination or compared with human examiners. Objective. To quantify human EMS evaluator inter-rater reliability and evaluate an AI video-analysis platform against it. Methods. In a prospective, fully crossed study, five certified EMS evaluators and an AI platform independently scored identical video-recorded EMT performances of cervical collar application (n=15), bag-valve-mask (BVM) ventilation (n=14), and medical assessment (n=15) on dichotomous checklists with critical-failure criteria. Agreement was assessed at item, score, and decision levels using Fleiss' kappa, Krippendorff's alpha, Gwet's AC1, and ICC(2,1)/ICC(2,k). Results. Human item agreement was moderate (kappa 0.409 to 0.467), as was single-rater reliability (ICC(2,1) 0.539 to 0.694), against good panel reliability (ICC(2,k) 0.854 to 0.919). Recorded pass/fail agreement was fair (kappa 0.297 to 0.388) and critical-failure agreement near zero for two skills (kappa 0.028, 0.119). AI alignment tracked rubric observability rather than task complexity: r = 0.857 (collar, exceeding every human), -0.173 (BVM), 0.664 (medical), and it was most lenient on two skills. Conclusions. Human evaluators are an imperfect standard, especially on critical failures. The AI was a legitimate additional rater where checklist items were discrete and visually verifiable, but not where credit required judging continuous quantities such as ventilation rate, volume, or suction duration. Defensible uses are formative and archival, not summative. These results reflect an early, non-specialist configuration: a baseline, not a limit.
Xu, Y.; Peng, Y.; Dan, L.; Mei, L.; Han, X.; Duan, P.
Show abstract
Abstract Background: Large language models (LLMs) are increasingly explored as adjunct tools in graduate medical education, but their reliability for answering discipline-specific certification examinations remains incompletely characterized. In dentistry in China, the national dental examination serves as a milestone for stomatology trainees. Prior studies predominantly rely on single metric, limiting reproducibility and granularity. Objective: To systematically evaluate the performance of five LLMs in answering the national standardized training examination for dental residents. Method: A total of 1,318 questions covering the six dentistry disciplines were selected. Five LLMs (DeepSeek-V3, DeepSeek-R1, Qwen3-32B, Claude 3.5, and GPT-4o) were evaluated via a single-response protocol on the Dify-based Chatflow framework. Performance combined human verification with automated scoring using the Retrieval-Augmented Generation Systems (RAGAS) framework across seven dimensions. Tasks involving question classification, key-point categorization, and instructional requirement summarization were designed to enable a horizontal comparison of models' cognitive capabilities, assessed using inter-model consistency scores. Results: DeepSeek-R1 achieved the highest correct choice rate (80.90%, p < 0.001) and answer correctness score (0.853, p < 0.001), significantly outperforming the others. All models exhibited limitations, including information insufficiency, redundancy, and hallucination-related issues (lowest 44.20% in DeepSeek-R1). In classification and summarization tasks, Claude 3.5 showed higher consistency scores (186/181) than Qwen3-32B (152/151). Conclusion: Evaluated LLMs demonstrated good performance on dental examination questions and in executing tasks. DeepSeek-R1 consistently outperformed the others. While LLMs showed potential as supportive tools in dental education, further refinement and validation are needed before formal adoption in graduate medical educational and clinical training settings.
MARTINEAU, R.; COMBES, C.; YAUY, K.; DERUELLE, P.
Show abstract
Background: Artificial intelligence (AI) is increasingly used in healthcare, but data on its real-world use and perceptions among obstetricians and gynecologists remain limited. Objectives: To assess familiarity, use, perceptions, and expectations regarding AI among physicians practicing obstetrics and gynecology in France. Study Design: We conducted a cross-sectional survey between June 2 and July 30, 2025, mainly distributed through professional organizations or associations and social media. Participation was voluntary and self-selected. AI use was defined as any use of an AI tool in routine clinical practice, regardless of type or frequency. Comparisons were made between AI users and non-users and between junior and senior physicians. Results: Among 469 respondents, representing approximately 5% of the target population, 199 of 466 (42.7%) reported AI use in practice; three respondents were excluded from this analysis because of discordant questionnaire responses. AI use was associated with male gender and professional status, with residents more represented among users. Junior physicians reported greater AI use than senior physicians and expressed greater concern regarding medical confidentiality. Current applications primarily involved administrative tasks, medical writing, literature retrieval, and translation; anticipated future uses centered on clinical decision support and medical data interpretation. Perceptions of several AI applications differed between groups. Conclusions: AI was reported to be used by a substantial proportion of respondents, primarily for supportive and informational tasks. These findings support further evaluation of training, validation, governance, and safe implementation of AI in obstetrics and gynecology.
Burlov, N.; Baranovskii, M.; Burlova, E.; Slavenko, M.; Khrykov, G.
Show abstract
Background. Journal clubs (JCs) are a popular education format. Interest in studying their impact is high, and authors often report positive results related to subjective parameters. Objective assessments of effectiveness are limited and contradictory. In this paper, we share our experience and describe our journal club effectiveness. Methods. We conducted a prospective cohort study within our online journal club. Meetings followed a discussion-based format and were held via Zoom, with timing and topics determined by voting in the club Telegram chat. Enrolment occurred in waves and included an application, entry test, and interview. During each recruitment wave, both club members (treatment group) and applicants (control) completed an admission test assessing knowledge of evidence-based medicine and statistics. Results. The JC currently comprises 27 members. Over the past year, 76 meetings were held, with 75% of participants grading their experience with 9 or 10 on a ten-point scale. Multivariate analysis demonstrated non-significantly results (SMD = 0.19 (95% CI 0.004; 0.38), p = 0.046) among participants. However, in other adjusted models, differences between groups were not statistically significant (p > 0.05). Conclusion. While the analysis of the subjective outcomes is consistent with findings from previous studies, the objective outcomes remain inconclusive. Further research is needed to refine the methodology for the organization and evaluation of journal clubs.
Dai, N.; Waarala, K. L.
Show abstract
Research Objectives: To evaluate whether publicly available institutional guidance supports patient-information decisions across scenarios and schools, and characterize document synthesis and evaluation consistency. Methods: We conducted an exploratory, vignette-based document analysis of a geographically diverse nonprobability sample of 20 U.S. osteopathic medical schools. Eight educational vignettes yielded 160 school-vignette pairs. Each pair underwent three separate AI-assisted retrieval-and-evaluation runs, classifying decision support as Explicitly Supported, Inferable, Ambiguous, or Not Addressed. Response selection prioritized greater support for discordant pairs, followed by fewer contributing documents and run order. One investigator verified or revised selected discordant classifications against cited evidence. Explicitly Supported and Inferable were grouped post hoc as sufficient decision support. Analyses were descriptive and included an exploratory two-school model-investigator comparison. Results: At least one eligible source was retrieved for 159 of 160 pairs (99.4%). Final classifications were Explicitly Supported for 18 pairs (11.3%), Inferable for 3 (1.9%), Ambiguous for 132 (82.5%), and Not Addressed for 7 (4.4%). Sufficient support occurred in 21 pairs (13.1%), most frequently for generative AI-assisted reflective writing (7/20 schools, 35%), and in none for personal cloud notes or official clinical logs. Ten schools had no sufficiently supported vignette; the maximum was four of eight. Multiple documents contributed to 111 evaluations (69.4%). Three-run ratings were unanimous for 113 pairs (70.6%), with 80.2% pairwise exact agreement. Investigator review retained 44 of 47 selected discordant ratings and revised three upward. In the two-school comparison, the investigator more frequently judged evidence sufficient when models judged it insufficient than the reverse. Conclusions: Relevant public guidance was frequently retrieved, but few school-vignette pairs were classified as providing sufficient scenario-specific decision support. These findings highlight a gap between identifying relevant guidance and determining an appropriate course of action within this evaluation framework. They support institutional review of how student-facing materials explain information use, storage, sharing, and approval requirements. Vignette-based review identifies questions requiring clarification. Evaluation with learners and assessment of internal and clinical-site guidance would help determine how these findings translate to students' decisions.
D'Amico, M.; Foster, D.; Mbapah, L.; Richman, M.
Show abstract
Introduction Inappropriate use of echocardiograms and stress tests with nuclear components in the evaluation of cardiopulmonary complaints (chest pain, dyspnea/shortness of breath (SOB), dizziness, and syncope) is common, despite Cardiology society guidelines for appropriate use. Such overuse raises healthcare costs, impairs the availability of resources for other patients, and exposes patients to radiation unnecessarily. Our Emergency Department (ED) runs an observation unit that sees thousands of patients annually with cardiopulmonary complaints. We observed many have echocardiograms and stress tests with nuclear components without indications for either. Working with our Department of Cardiology, we promoted guidelines for appropriate use of echocardiograms and stress tests with nuclear components in the evaluation of such patients. Methods We collected 2024 and 2025 data regarding OU admissions. Patents were included if they were [≥]18 years, in the ED OU between 1/1/24 and 12/31/25, aad an OU admission complaint of chest pain, dyspnea/SOB, dizziness, and syncope. We calculated the percent of patients with such presentations who received either an echocardiogram or a stress test with a nuclear component. We then compared the 2024 (pre-implementation) data with full-year data from the first full year of implementation (2025). Results After introducing educational guidelines, there were statistically-significant reductions in nuclear stress tests ordered for dyspnea/SOB (absolute reduction 9.3%; p = 0.0009) and overall usage of nuclear stress tests (absolute reduction 2.4%; p = 0.0095). This was offset by increases in echocardiograms ordered for chest pain (absolute increase 3.7%; p = 0.005) and overall usage of echocardiograms (absolute increase 2.0%; p = 0.0454). There was no net change in the overall use of echocardiograms and nuclear stress tests when considered together: 85.2% of patients had one or the other study in 2024 vs. 84.8% in 2025; p = 0.5768. Discussion Introducing and promoting appropriate-use guidelines for echocardiograms and stress tests with nuclear components in the evaluation of cardiopulmonary complaints was associated with a reduction in unnecessary nuclear stress tests, but not echocardiograms. Success of this program to reduce nuclear stress tests may be attributed to several factors. First, the relevant specialties (emergency medicine, cardiology) were engaged. Second, we relied on guidelines adopted by professional organizations. Third, the initiative corresponds to nationwide efforts to decrease cost of care by providing evidence-based diagnostic and treatment management. Failure to reduce echocardiograms may be due to a well-documented multi-year time lag between changes in medical knowledge and wide-scale adoption; failure to develop the guidelines in collaboration with house staff and mid-level providers and promote them at multiple times and venues in a multi-modal manner; and utilize Department leadership to promote their usage.
Haupt, S.; Parkiet, T.; Mirza, I.; Webster, K.; Waise, M.; Wainer, Z.; Kwan, K.; Billiards, S.; Graham, B.; Tannenbaum, C.; Huxley, R.; Lamon, S.
Show abstract
Objective: To investigate and establish a baseline for sex and gender considerations in policy, research and curricula across the state of Victoria. Design and setting: Victoria was selected as a case study for Australia, using a mixed-methods approach to examine health and medical university curricula, research organisation policies and research funding between 2020-2025 prior to mandated inclusion. Main outcome measures: Primary outcomes include identification of predefined sex- and gender-related terms in university curricula descriptors and funded grant descriptions; and questionnaire responses from university course coordinators and organisational leads. Results: Data mining across nine Victorian universities (318 courses/3383 units) identified ~93% of units and ~60% of healthcare courses lacked sex and gender terms in their descriptors. Among medical research organisations operating in Victoria, including peak bodies, research institutes, hospitals and universities, ~70% (18/26) of the survey responders reported having no sex and gender policy. Rates of sex- and gender-term inclusion in research grants allocated in Victoria (3388) and Australia-wide (8974) validated Victoria as a case study for Australia for National Health and Medical Research Council (9.5%/8.9% respectively), Medical Research Future Funds (11%/10.2%), and Australian Research Council (4.8%/4.1%). One in nine Victorian awards from five government initiatives and one in ten from 12 non-government agencies included sex- and gender-term related terms in their guidelines. Conclusions: These baseline metrics indicate that sex and gender are still not widely considered in the education and research ecosystems. These findings support the need to build inclusive research policy at a national and state level, and accreditation standards across university education.
Dali, A. M.; Vujicic, A.; Gato, D.; Volkan, E.
Show abstract
Abstract Background: Cyprus has transitioned into a high-alert zone for mosquito-borne viruses. This necessitates a proactive assessment of its future physicians. Following the 2022 re-establishment of Aedes aegypti in Larnaca and Aedes albopictus in Limassol, Cyprus, the island is now biologically primed for local outbreaks of dengue, Zika, and chikungunya, alongside Culex-borne West Nile virus. The study assessed medical students' knowledge and risk perception. Methods: A cross-sectional online survey administered the validated MosquitoWise Health Belief Model instrument to undergraduate-entry and graduate-entry students at a university in Cyprus in mid-2026. Results: Of 1023 students, 119 responded (11.6%), with pre-clinical students overrepresented (p<0.001). Mean knowledge was 5.43/9, stronger on mosquito biology (72.5%) than virus knowledge (54.2%). Median severity was 6.00/7 against susceptibility 4.25/7. Knowledge differed by study stage (p<0.001), correlated with severity (rs = 0.399) but not susceptibility. Conclusions: The study establishes a critical baseline for pre-clinical medical students in Cyprus. Overall, participants performed better on mosquito biology when compared to mosquito-borne viruses and how they spread. Our findings highlight the value of integrating theoretical microbiology with clinical practice in medical education to enhance physicians' preparedness for the changing landscape of newly emerging infections in Cyprus and across Europe as a whole. Key Words: Arboviruses, Cross-Sectional Studies, Cyprus, Health Belief Model, Medical Students, Risk Perception.